Improving Negative Sampling for Word Representation using Self-embedded Features

Although the word-popularity based negative sampler has shown superb performance in the skip-gram model, the theoretical motivation behind oversampling popular (non-observed) words as negative samples is still not well understood. In this paper, we start from an investigation of the gradient vanishing issue in the skipgram model without a proper negative sampler. By performing an insightful analysis from the stochastic gradient descent (SGD) learning perspective, we demonstrate that, both theoretically and intuitively, negative samples with larger inner product scores are more informative than those with lower scores for the SGD learner in terms of both convergence rate and accuracy. Understanding this, we propose an alternative sampling algorithm that dynamically selects informative negative samples during each SGD update. More importantly, the proposed sampler accounts for multi-dimensional self-embedded features during the sampling process, which essentially makes it more effective than the original popularity-based (one-dimensional) sampler. Empirical experiments further verify our observations, and show that our fine-grained samplers gain significant improvement over the existing ones without increasing computational complexity.Comment: Accepted in WSDM 201

Learning to Rank Question Answer Pairs with Holographic Dual LSTM Architecture

We describe a new deep learning architecture for learning to rank question answer pairs. Our approach extends the long short-term memory (LSTM) network with holographic composition to model the relationship between question and answer representations. As opposed to the neural tensor layer that has been adopted recently, the holographic composition provides the benefits of scalable and rich representational learning approach without incurring huge parameter costs. Overall, we present Holographic Dual LSTM (HD-LSTM), a unified architecture for both deep sentence modeling and semantic matching. Essentially, our model is trained end-to-end whereby the parameters of the LSTM are optimized in a way that best explains the correlation between question and answer representations. In addition, our proposed deep learning architecture requires no extensive feature engineering. Via extensive experiments, we show that HD-LSTM outperforms many other neural architectures on two popular benchmark QA datasets. Empirical studies confirm the effectiveness of holographic composition over the neural tensor layer.Comment: SIGIR 2017 Full Pape

Neural Conversation Generation with Auxiliary Emotional Supervised Models

An important aspect of developing dialogue agents involves endowing a conversation system with emotion perception and interaction. Most existing emotion dialogue models lack the adaptability and extensibility of different scenes because of their limitation to require a specified emotion category or their reliance on a fixed emotional dictionary. To overcome these limitations, we propose a neural conversation generation with auxiliary emotional supervised model (nCG-ESM) comprising a sequence-to-sequence (Seq2Seq) generation model and an emotional classifier used as an auxiliary model. The emotional classifier was trained to predict the emotion distributions of the dialogues, which were then used as emotion supervised signals to guide the generation model to generate diverse emotional responses. The proposed nCG-ESM is flexible enough to generate responses with emotional diversity, including specified or unspecified emotions, which can be adapted and extended to different scenarios. We conducted extensive experiments on the popular dataset of Weibo post--response pairs. Experimental results showed that the proposed model was capable of producing more diverse, appropriate, and emotionally rich responses, yielding substantial gains in diversity scores and human evaluations.Peer reviewe

Cross-Language Question Re-Ranking

We study how to find relevant questions in community forums when the language of the new questions is different from that of the existing questions in the forum. In particular, we explore the Arabic-English language pair. We compare a kernel-based system with a feed-forward neural network in a scenario where a large parallel corpus is available for training a machine translation system, bilingual dictionaries, and cross-language word embeddings. We observe that both approaches degrade the performance of the system when working on the translated text, especially the kernel-based system, which depends heavily on a syntactic kernel. We address this issue using a cross-language tree kernel, which compares the original Arabic tree to the English trees of the related questions. We show that this kernel almost closes the performance gap with respect to the monolingual system. On the neural network side, we use the parallel corpus to train cross-language embeddings, which we then use to represent the Arabic input and the English related questions in the same space. The results also improve to close to those of the monolingual neural network. Overall, the kernel system shows a better performance compared to the neural network in all cases.Comment: SIGIR-2017; Community Question Answering; Cross-language Approaches; Question Retrieval; Kernel-based Methods; Neural Networks; Distributed Representation

Post-capitalist property

When writing about property and property rights in his imagined post-capitalist society of the future, Marx seemed to envisage ‘individual property’ co-existing with ‘socialized property’ in the means of production. As the social and political consequences of faltering growth and increasing inequality, debt and insecurity gradually manifest themselves, and with automation and artificial intelligence lurking in the wings, the future of capitalism, at least in its current form, looks increasingly uncertain. With this, the question of what property and property rights might look like in the future, in a potentially post-capitalist society, is becoming ever more pertinent. Is the choice simply between private property and markets, and public (state-owned) property and planning? Or can individual and social property in the (same) means of production co-exist, as Marx suggested? This paper explores ways in which they might, through an examination of the Chinese household responsibility system (HRS) and the ‘fuzzy’ and seemingly confusing regime of land ownership that it instituted. It examines the HRS against the backdrop of Marx’s ideas about property and subsequent (post-Marx) theorizing about the legal nature of property in which property has come widely to be conceptualized not as a single, unitary ‘ownership’ right to a thing (or, indeed, as the thing itself) but as a ‘bundle of rights’. The bundle-of-rights idea of property, it suggests, enables us to see not only that ‘individual’ and ‘socialized’ property’ in the (same) means of production might indeed co-exist, but that the range of institutional possibility is far greater than that between capitalism and socialism/communism as traditionally conceived

Financial Security and Optimal Scale of Foreign Exchange Reserve in China

The study of how foreign exchange reserves maintain financial security is of vital significance. This paper provides simulations and estimations of the optimal scale of foreign exchange reserves under the background of possible shocks to China’s economy due to the further opening of China’s financial market and the sudden stop of capital inflows. Focused on the perspective of financial security, this article tentatively constructs an optimal scale analysis framework that is based on a utility maximization of the foreign exchange reserve, and selects relevant data to simulate the optimal scale of China’s foreign exchange reserves. The results show that: (1) the main reason for the fast growth of the Chinese foreign exchange reserve scale is the structural trouble of its double international payment surplus, which creates long-term appreciation expectations for the exchange rate that make it difficult for international capital inflows and excess foreign exchange reserves to enter the real economic growth mechanism under the model of China’s export-driven economy growth; (2) the average optimal scale of the foreign exchange reserve in case of the sudden stop of capital inflows was calculated through parameter estimation and numerical simulation to be 13.53% of China’s gross domestic product (GDP) between 1994 and 2017; (3) with the function of the foreign exchange reserves changing from meeting basic transaction demands to meeting financial security demands, the effect of the foreign exchange reserve maintaining the state’s financial security is becoming more and more obvious. Therefore, the structure of foreign exchange reserve assets should be optimized in China, and we will give full play to the special role of foreign exchange reserve in safeguarding a country’s financial security

Higher Education Input, Technological Innovation, and Economic Growth in China

Based on the theoretical analysis of the relationship between China’s higher education input, technological innovation, and economic growth, this paper chooses the 1997–2015 sample data of China, and uses a vector auto regression (VAR) model to test the relationship between the three. The results show that educational input, technological innovation, and economic growth form an interaction mechanism featuring dynamic circulation. Higher education input and technological innovation are two important factors influencing economic growth. In the meantime, higher education input is an important source and driving force of technological innovation, and technological innovation will further promote economic growth. However, technological innovation has a delayed positive effect on economic growth, so higher education input demands a long-term view and thinking for quick success, and instant benefits should be avoided

A study on the influence mechanism of CBDC on monetary policy: An analysis based on e-CNY.

This paper attempts to introduce central bank digital currency (CBDC) into the analysis framework of monetary policy, and studies the influence mechanism of e-CNY, central bank digital currency in China, on the monetary policy of the central bank from the aspects of money demand, money supply and monetary policy transmission mechanism. The research finds that e-CNY will have significant impact on monetary policy: (1) E-CNY will change the structure of money demand, speed up currency circulation, make central bank reserves more controllable and money supply more intelligent; (2) E-CNY will increase the volatility and expansion effect of currency multiplier to a certain extent; (3) E-CNY will dredge the transmission channel of monetary policy so as to improve the transmission effect of existing monetary policy tools. At the same time, based on the organic combination with structural monetary policy tools, it will achieve precise implementation of medium-term lending facilities (MLF), pledged supplementary lending (PSL), and it may bring new monetary policy tools. (4) E-CNY will make the intermediate target of monetary policy more controllable and reliable, and have a positive impact on the target of monetary policy through the smooth transmission of monetary policy channels. Therefore, it is necessary to strengthen the research on CBDC, give full play to the positive role of e-CNY in monetary policy, and improve the effectiveness of monetary policy